It is known that there are methods and models to stabilize video frames captured at a constant frame rate. Stabilization is implemented to reduce or remove vibrations on the image sequence, for example caused by the shaking hands of a camera user. This kind of a jitter or vibration reduction is crucial since such vibrations disturb the viewing experience; reduce image quality and the performance of possible subsequent image processing modules. Stabilization methods provide means of computing the jitter related transformation between two video frames and compensate this movement by warping the current frame accordingly. Even the video sequences captured with fixed line of sight cameras may contain “jitter” due to various environmental factors, such as wind, affecting the camera platform or mount. Stabilization methods may also be applied to pre-recorded videos or image sequences in addition to the live video acquisition.
In currently used methods, the image sequence or image frames are generally captured by image sensors consisting two dimensional arrays of pixels. Depending on the sensor, the images might be coloured or greyscale. The output of infrared (IR) cameras can be considered as greyscale. Essentially, stabilization algorithms register every frame obtained from the image source with a reference frame. They accomplish this, generally, by computing the parameters of the assumed transformation between the two frames. This transformation can be purely translational with two parameters. Alternatively, more complex models such as affine or perspective with 6 and 8 parameters, respectively, can be employed. The transformation that the stabilization method utilizes should be chosen depending on the application.
Apart from registering the reference and current frames, the update of the reference frame is a necessity; especially if the stabilization will be deployed on a system which is supposed to work for a long time without interruption. This necessity occurs even when the camera has a fixed line of sight. The necessity stems from the fact that when the system works for a prolonged time, permanent shifts might occur in the line of sight. Furthermore, since almost all registration methods work on the color/grayscale similarity between the reference and current frames, any kind of illumination changes in the environment (or temperature change in the case of infrared cameras) would deteriorate this similarity. Thus the reference frame has to be updated regularly to ensure that it reflects the current state of the environment.
The United States patent document US2006274156, an application in the state of the art, discloses an image sequence stabilization method and a camera, wherein a sequence of input digital images are captured and replicated to provide a corresponding sequence of archival images and a corresponding sequence of display images.
The said method resembles the present invention in that it can perform stabilization using pixel projection correlations. But the present invention differs from the said method in terms of optimizations for a fixed line of sight camera.
The Korean patent document KR20100050906, an application in the state of the art, discloses a method for stabilizing an image sequence and a method for updating the reference frame according to the magnitude of the motion vector.
The present invention differs from the said method in terms of the update technique of the reference frame. Present invention does not only decide whether to update reference frame according to the magnitude of the motion vector but also determines “how” to update it.